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A  SOURCE  SELECTION  DECISION  SUPPORT  SYSTEM 


Significant  amounts  of  resources  are  allocated  to  Source  Selection 
Evaluations  within  the  Department  of  Defense.  A  typical  source 
selection  is  divided  into  sub-components  such  as:  technical  criteria, 
management  criteria,  and  cost.  Source  selections  are  completed  for  a 
variety  of  systems  including,  but  not  limited  to,  aircraft,  missiles, 
ships,  space  systems,  system  level  depot  maintenance  {public  vs.  private 
competitions) ,  and  computer  equipment.  Currently,  the  procedure  for 
selecting  the  winner  involves  an  assessment  of  all  the  criteria  and  sub¬ 
elements  on  which  the  contractors  have  been  evaluated.  Teams 
responsible  for  each  sub-component  brief  the  Source  Selection  Evaluation 
Board  on  the  various  results,  and  a  winner  is  selected. 

We  have  developed  a  prototype  Decision  Support  System  in  Microsoft  EXCEL 
that  can  be  used  to  compile  source  selection  information  from  the 
various  teams  (technical,  management,  cost,  etc.)  in  order  to  make  an 
objective  selection  based  on  the  analysis  of  each  of  the  criteria  and 
sub-criteria.  The  Decision  Support  System  utilizes  a  weighted 
evaluation  mathematical  model  to  compile  various  contractor  scores  for 
sub-criteria. 

This  Decision  Support  System  demonstrates  a  methodology  that  can  be  used 
by  management  and  analysts  in  completing  source  selections. 
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I. 


INTRODUCTION 


The  source  selection  process  within  the  Department  of  Defense  is  one  of 
the  most  time  consuming  activities  for  acquisition  personnel.  A  source  selection 
team  typically  consists  of  a  technical  team,  management  team  and  cost  team. 
Team  members  are  tasked  with  evaluating  the  competing  contractor's  proposals 
in  their  area  of  expertise  (i.e.  technical,  management  or  cost). 

Currently  the  evaluation  method  for  competitions  is  to  select  one 
contractor  for  award  on  the  basis  of  which  proposal  represents  the  best  value  to 
the  Government.  These  proposals  are  evaluated  using  evaluation  factors  such 
as:  Technical,  Management,  Cost,  and  Integrated  Logistics  Support.  The 
evaluation  factors  vary  in  importance  for  each  unique  competition. 

There  are  many  complexities  that  exist  in  source  selections:  numerous 
contractors,  various  evaluation  criteria,  and  different  rankings  for  each  of  the 
evaluation  criteria.  Compiling  the  results  of  each  team's  evaluation  into  an 
understandable  format  to  present  to  top  management  is  a  difficult  process. 

A  Computer  Based  Decision  Support  System  (DSS)  may  help  to 
overcome  several  of  the  obstacles  which  currently  exist  in  the  evaluation  of 
contractor  bids.  This  paper  presents  a  DSS  which  implements  a  weighted 
evaluation  model  (the  Brown-Gibson  model)  in  order  to  summarize  source 
selection  information. 

The  objective  of  the  DSS  is  to  provide  an  automated  system  that  can 
function  as  a  tool  to  aid  decision  makers  in  the  evaluation  of  source  selections, 
and  to  provide  a  tool  which  visually  assists  the  evaluation  team  in  summarizing 
the  subjective  and  objective  attributes  of  the  competition. 

II.  THE  SOURCE  SELECTION  DECISION  SUPPORT  SYSTEM 

This  Decision  Support  System  summarizes  source  selection  information 
by  generating  a  spreadsheet  that  is  created  using  macros  and  various  user 
inputs.  Information  from  the  technical,  management  and  cost  teams  is  used  to 
calculate  an  overall  score  for  each  contractor  participating  in  the  source 
selection.  Much  of  the  information  that  is  entered  into  the  spreadsheet  comes 
from  the  Request  for  Proposal  (RFP).  Each  RFP  contains  guidelines  on  how 
the  contractors  will  be  evaluated.  As  mentioned  in  the  introduction,  the  most 
common  evaluation  criteria  are  technical,  management  and  cost.  Also  included 


in  the  RFP  are  sub-elements  for  each  of  the  main  criteria.  For  example,  there 
may  be  5  sub-elements  which  are  analyzed  for  the  technical  criteria,  and  3  sub¬ 
elements  for  the  management  criteria.  This  information  from  the  RFP  is 
necessary  in  order  to  complete  a  "session"  using  the  Source  Selection  DSS. 

The  OSS  is  built  using  Microsoft  Excel  4.0,  and  is  designed  to  run  on  a 
386  based  IBM  compatible  computer  with  a  hard  disk  drive  and  color  monitor. 
The  DSS  is  designed  for  someone  familiar  with  DoD  and  the  basic  concepts 
involved  in  conducting  a  source  selection,  however,  even  an  inexperienced  user 
can  manipulate  the  model.  Currently,  the  DSS  can  accommodate  up  to  12 
companies  in  a  source  selection. 

III.  WHAT  IS  A  COMPUTER  BASED  DECISION  SUPPORT  SYSTEM  ? 

A  Computer  Based  Decision  Support  System  is  a  computer  driven  system 
designed  to  aid  managers  in  making  decisions.  Some  of  the  major 
characteristics  of  a  DSS  are:  fast  response,  ability  to  conduct  sensitivity 
analyses,  objective  decision  making,  and  facilitated  communication.  There  are 
three  basic  components  in  a  DSS: 

•  Dialog  Base 

•  Model  Base 

•  Data  Base 

Each  of  the  above  elements  and  how  they  relate  to  the  Source  Selection 
DSS  will  be  described  in  more  detail. 

A.  DIALOG  BASE  COMPONENT 

The  dialog  base  component  of  the  DSS  consists  of  macros,  buttons  and 


dialog  boxes  written  in  Microsoft  Excel.  Each  of  these  features  provides  the 

~LCJ 

DSS  user  with  a  comfortable  environment  that  allows  interaction  and  real-time  W 
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response.  The  screens  are  designed  to  be  clear  and  concise,  and  the  user  is  □ 

allowed  to  move  around  by  clicking  on  buttons  that  are  available  on  each 
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screen.  There  are  buttons  to  move  forward,  return  to  the  previous  screen,  or  to  | 

access  help  menus. 
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Macros  are  lines  of  software  code  which  are  instructions  that  tell  the 
computer  to  invoke  certain  operations.  At  their  best,  macros  can  do  in  minutes 
what  might  take  hours  to  do  with  standard  worksheets  and  processes.  An 
example  of  one  macro  used  in  this  DSS  is:  a  macro  builds  a  spreadsheet  that 
the  user  then  enters  data  into. 

The  dialog  base  is  the  guiding  force  behind  the  model  base  and  the 
database.  More  importantly,  it  is  the  interface  with  the  user,  so  it  must  be  easy 
to  use  and  understand.  Excel  offers  several  ways  to  request  information  from 
users.  The  dialog  base  uses  the  option  of  creating  a  dialog  box.  Excel  returns 
the  information  a  user  enters  into  the  dialog  box  to  the  macro  sheet  that  brought 
the  dialog  box  up  on  the  screen.  The  dialog  box  allows  the  user  to  easily  enter 
their  source  selection  inputs  into  the  DSS.  This  is  accomplished  by  prompting 
the  user  with  questions  and  stepping  them  through  the  process  of  entering  the 
information  required  to  build  the  competition  spreadsheet. 

B.  MODEL  BASE  COMPONENT 

The  model  base  component  of  this  DSS  consists  of  a  model  called  the 
Brown-Gibson  Model,  which  is  a  model  used  to  conduct  a  weighted  evaluation 
for  objective  and  subjective  measurements.  It  was  mentioned  earlier  in  the 
report  that  the  goal  of  the  DSS  is  to  assist  top  management  within  the 
Department  of  Defense  in  making  decisions  during  the  source  selection  process. 
The  Brown-Gibson  Model  combines  the  objective  and  subjective  measures  from 
the  source  selection  evaluation  and  calculates  a  numeric  score  for  each 
alternative.  These  numeric  scores  can  then  be  analyzed  to  determine  which 
alternative  should  be  selected. 

The  Brown-Gibson  Model  is  used  to  summarize  the  technical  and 
management  (subjective)  criteria  and  the  cost  (objective)  criteria.  The  Brown- 
Gibson  Model  can  be  summarized  in  a  mathematical  expression: 

Weighted 

Evaluation  ■  (WT^SM,)  +  (WT^SMj)  +  (WTjXOM,) 
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Where: 


WTn  = 

the  weight,  or  importance,  of  each  individual  criteria,  i.e. 
cost,  technical,  and  management 

SMn  = 

the  subjective  measure  of  criteria  n 

OMn  = 

the  objective  measure  of  criteria  n 

This  formula  can  be  extended  for  as  many  subjective  elements  as  there  are  in 
the  source  selection.  It  was  mentioned  in  the  introduction  that  many  source 
selections  are  evaluated  on  cost,  technical  and  management  criteria.  Therefore, 
these  are  the  criteria  that  were  used  in  the  examples. 

1.  Subjective  Measures 

Determining  the  subjective  measure  for  each  alternative  involves 
transforming  the  nominal  ratings  into  numerical  scores.  Then  each  individual 
attribute  within  a  subjective  category  is  assigned  a  weighting.  (NOTE:  These 
weightings  must  sum  to  1). 

The  calculation  of  the  subjective  measure  for  each  alternative  is  as 
follows: 

N 

SM„  =  X( subjective  attribute  weight)  *  subjective  evaluation  rating- 
n=1  £  subjective  evaluation  rating, 

alii 

The  subjective  attribute  weight  is  the  relative  importance  for  each  of  the 
criteria  within  this  particular  subjective  category.  The  evaluation  ratings  are  the 
nominal  point  values  which  were  determined  by  the  source  selection  team  as  to 
how  a  company  should  be  rated.  The  most  common  rankings  for  source 
selections  consist  of  5  possibilities: 
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•  outstanding  (5) 

•  highly  satisfactory  (4) 

•  satisfactory  (3) 

•  marginal  (2) 

•  unsatisfactory  (1) 

An  example  will  be  presented  next  in  an  effort  to  clarify  the  model  calculations. 

This  example  presents  a  situation  in  which  three  companies  are 
evaluated  on  the  basis  of  the  technical  criteria.  The  management  criteria  would 
be  evaluated  in  a  similar  fashion.  It  was  mentioned  earlier  in  the  paper  that 
there  are  often  sub-elements  within  an  evaluation  criterion.  The  example  below 
shows  5  sub-elements  under  the  technical  attribute.  These  items  would  be 
specified  in  the  RFP.  For  this  example,  equal  weights  were  assumed  for  each 
sub-element.  The  values  shown  for  each  company  represent  an  example  of  how 
the  technical  team  could  rate  each  company  for  each  sub-element,  where  the 
values  of  1-5  correspond  to  the  rankings  listed  above. 


Technical 

Subjective 

Attribute 

Weight 

Company  1 

Company  2 

Company  3 

Sum 

Production 

0.20 

5 

3 

3 

11 

Quality 

0.20 

2 

4 

5 

11 

Facilities 

0.20 

1 

3 

3 

7 

Safety 

0.20 

4 

4 

4 

12 

Engineering 

0.20 

2 

5 

3 

10 

1.00  SMi=  0.2625  SM?*  WSKMSKKSk  SM3=  0.3578 
The  subjective  measures  (SM,  through  SM3)  were  calculated  using  the 
formula  for  SMn  shown  above.  The  calculations  for  SM,  are  included  next  as  an 
example: 

SM1  =  (.2)(5/11)  +  (.2)(2/i1)  +  (.2)(1/7)  +  (.2)(4/12)  +  (.2)(2/10) 

=  .2625 

The  subjective  measures  for  Company  2  and  Company  3  were  also  calculated 
using  the  same  formula. 

The  subjective  portion  of  the  model  base  interacts  with  both  the  dialog 
base  and  the  database  of  the  DSS.  The  user  is  initially  asked  a  series  of 
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questions,  and  the  responses  are  stored  to  be  used  in  the  calculation  section  of 
the  Brown  Gibson  Model.  The  user  is  asked  to  input  what  subjective  criteria  are 
being  evaluated,  the  weightings  for  the  criteria  and  the  actual  nominal  scores. 
This  information  is  then  saved  into  the  database  for  future  use. 

The  above  table  demonstrates  some  of  the  calculations  which  are 
completed  in  the  model.  The  table  above  would  be  built  by  macros  as  a  result  of 
the  user  inputting  information  into  dialog  boxes. 

2.  Objective  Measures 

Determining  the  objective  measure  (OM)  for  each  alternative  involves 
transforming  the  cost  into  a  relative  score  between  0  and  1 ,  so  that  the  sum  for 
all  alternatives  is  1 .0.  The  methodology  that  is  used  to  transform  the  costs  into 
values  between  0  and  1  is  presented  in  the  equations  below: 

OMn  =  1/(Cost„  *S) 

N 

S  =  I  1/Cost„ 

n-1 

An  example  of  how  the  OMn  is  calculated  is  shown  in  the  table  below. 

The  formulas  for  both  subjective  and  objective  measures  are  incorporated  into 
the  cost  section  of  the  DSS,  so  the  user  does  not  need  to  know  all  of  the  details 
to  use  the  system.  However,  for  explanation  purposes,  a  basic  knowledge  of 
how  the  objective  measures  are  calculated  is  important  for  the  user  to 
understand. 

ALTERNATIVE _ COST 

Company  1  $130,000 

Company  2  $150,000 

Company  3  $165,000 

S  =  1/130,000  +  1/150,000  +  1/165,000 

=  .000020418 
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OM, 

= 

1/(130,000  V00002041 8) 

— 

.377 

OM, 

= 

1/(150,000  *  .000020418) 

= 

.326 

om3 

= 

1/(165,000  *  .000020148) 

s 

.297 

As  a  check,  the  objective  measures  should  sum  to  1 .0:  .377  +  .326  +  .297  =  1 .0 

The  costs  above  are  the  result  of  the  cost  team  developing  an 
independent  estimate  based  on  each  company's  proposal.  The  highest  score 
goes  to  the  company  with  the  lowest  cost.  This  is  due  to  the  fact  that  low  cost  is 
often  a  criterion.  However,  cost  is  not  the  only  criterion  to  consider.  Therefore, 
there  is  a  need  to  tabulate  the  cost  results  into  decimal  format,  so  that  the  other 
evaluation  criteria  can  also  be  factored  in. 

3.  Combining  the  Measures 

Once  the  objective  and  subjective  measures  for  each  company  are 
calculated  within  the  categories,  the  overall  scores  for  the  companies  are 
calculated  by  applying  the  same  basic  principles.  The  user  must  determine  how 
the  various  categories  should  be  weighted  (i.e.  how  the  categories  are  ranked 
between:  technical,  management,  and  cost,  etc.).  They  can  all  have  an  equal 
weight,  or  different  weights  can  be  assigned  based  on  the  importance  of  the 
criterion.  It  is  important  to  keep  in  mind  that  all  weights  must  add  up  to  1 .0. 
Guidance  for  the  weights  can  be  found  in  the  RFP.  Typical  RFPs  will  not 
explicitly  state  weighting  factors,  but  rather  will  indicate  a  ranking  from  highest  to 
lowest  for  the  various  criteria.  An  example  of  wording  from  the  RFP  is: 
'Technical  is  the  most  important  criterion,  cost  is  the  second  most  important 
criterion  and  management  is  the  least  important  criterion,  with  management 
being  half  as  important  as  cost".  From  this  statement,  weightings  can  be 
assigned  for  each  of  the  criteria.  For  illustration  purposes,  it  was  assumed  that 
only  one  subjective  measure,  technical,  and  one  objective  measure,  cost,  were 
used  for  the  example  evaluation  presented  below. 
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Category 


Weight 


Company  1 


Company  2 


Company  3 


Cost 

Technical 


0.50 

0.50 

1.00 


0.3770 

0.2625 

0.3198 


0.3260 

0.3797 


0,2970 

0.3578 

0.3274 


Based  on  the  above  calculations,  company  2  would  be  selected  as  the  overall 


winner  in  the  competition,  even  though  they  would  not  be  selected  if  cost  were 
the  only  evaluation  factor. 


C.  DATA  BASE  COMPONENT 


The  data  base  component  of  the  Source  Selection  DSS  is  built  around  the 
concept  that  the  key  information  required  when  running  this  program  is  the  most 
recent  evaluation  data  available.  For  this  reason,  there  is  no  need  to  bring  in 
any  historical  tables,  nor  do  we  need  to  sort  through  prior  competitions  to  assign 
a  value  to  the  subjective  criteria. 

The  Source  Selection  DSS  incorporates  a  function  for  either  a  new  or  an 
existing  competition.  If  the  competition  is  new,  the  user  is  led  through  a  process 
of  queries  in  order  that  a  new  spreadsheet  can  be  assembled.  If,  however,  the 
user  wishes  to  run  a  previous  competition,  then  it  is  necessary  only  to  type  in  the 
file  name  and  that  competition  is  restored  from  the  data  base.  The  existing  files 
may  be  edited  with  new  information  or  the  user  can  conduct  sensitivity  analyses 
with  a  range  of  data.  For  new  users  the  existing  file  feature  is  an  excellent  tool 
for  familiarizing  themselves  with  the  various  aspects  of  the  Source  Selection 
DSS.  At  any  time  during  the  session,  the  user  may  select  the  help  function  by 
clicking  on  the  help  button.  The  help  buttons  can  be  used  as  a  guide  for 
rankings  or  for  more  knowledge  about  the  Brown-Gibson  calculations.  The  user 
is  then  free  to  scroll  throughout  the  help  function  and  learn  about  the  criteria 
employed. 

IV.  POSSIBLE  IMPLEMENTATION  PROBLEMS/ISSUES 


One  potential  difficulty  in  implementing  this  model  will  be  gaining  top 
management  support  for  the  model  to  be  used  as  a  decision  tool.  Currently,  the 
source  selection  decision  process  is  often  completed  by  the  senior  management 

within  the  command.  One  of  the  goals  in  developing  this  model  was  to  create  a 
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decision  tool  that  would  make  the  process  of  choosing  a  winner  in  a  source 
selection  a  more  "scientific"  process.  In  the  current  system,  the  Source 
Selection  Advisory  Council  is  provided  a  briefing  on  each  of  the  areas  being 
evaluated,  for  example:  technical,  management  and  cost.  The  technical  and 
management  briefings  (subjective  areas)  list  the  companies  and  the  results  of 
the  ranking  process  as  outstanding,  highly  satisfactory,  satisfactory,  marginal, 
and  unsatisfactory.  A  cost  briefing  is  also  given,  which  provides  an  objective 
measure.  Once  all  of  the  briefs  have  been  presented,  top  management  must 
make  a  decision.  Without  a  system  that  summarizes  the  information  in  an 
objective  manner,  decisions  may  tend  to  be  more  subjective. 

This  model  is  not  intended  to  replace  top  n  anagement's  decision 
authority.  There  is  an  inherent  weakness  with  the  inputs  of  the  model  because 
two  of  the  factors  are  subjective.  Hence  all  subjectivity  is  not  eliminated.  The 
need  for  top  management  to  determine  close  decisions  for  the  best  value  of  the 
government  still  exists.  The  Source  Selection  Model  is  designed  primarily  to  be 
used  as  a  tool  in  summarizing  all  of  the  information  collected  and  analyzed 
during  a  source  selection. 

V.  SUMMARY 

This  model  demonstrates  the  useful  qualities  of  a  Decision  Support 
System  and  its  applicability  to  the  source  selection  environment.  Although  this 
model  will  not  eliminate  all  of  the  subjectivity  associated  with  source  selections, 
it  does  require  that  scores  and  weightings  be  tabulated  for  each  sub-element  in 
the  evaluation.  This  may  help  the  participants  in  a  source  selection  put 
everything  in  perspective. 
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